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Use of the Significance Test as Protection 
Against Spuriously High Standardized Effect Sizes: 

Introduction of the Protected Effect Size 

The concept of effect size has become very important in educational research. Some 
have even advocated using effect size estimates in place of tests of statistical significance (e.g., 
Carver, 1993; Nix & Barnette, 1998; Schmidt, 1996). Cohen (1969, 1988) recommended 
specific levels of effect size for "small," "medium," and "large" effects. However, even Cohen 
acknowledged that these values are relative to the specific content and method in a given 
research situation. The purpose of this study is to determine the level of standardized effect sizes 
obtained by chance and the use of significance tests to guard against spuriously high 
standardized effect sizes. It introduces the concept of the “protected effect size.” 

Backgroxmd 

The concept of effect size has been aroxmd for many years. Cohen (1969) is generally 
credited with coining the term. However, the development of meta-analysis by Glass, Rosenthal 
and others in the 1970s (e.g.. Glass, 1976; 1978; Glass & Hakstian, 1969; Rosenthal, 1976, 1978) 
and the popularity of a book on meta-analysis in 1981 (Glass, McGaw, & Smith) are the catalysts 
for the interest in the concept. Numerous publications followed on applications of effect size 
methodology (e.g.. Lynch, 1987; McLean, 1983), methods for estimating effect size and its 
properties (e.g., Fowler, 1988; 1993; Gibbons, Hedeker, & Davis, 1993; Hedges, 1981, 1984; 
Huynh, 1989; Kraemer, 1983; Reichhardt & Gollob, 1987; Thomas, 1986), extracting effect size 
estimates from existing studies (e.g.. Hedges, 1982; Snyder & Lawson, 1993), and correcting 
effect size estimates (Snyder & Lawson, 1993). Wolf (1986) presented a general methodology 
for conducting meta-analysis including the extraction and testing of effect sizes. 

Perhaps no one has had a greater impact on the use of effect sizes than Cohen (1977, 
1988) through his books on power analysis. In these books, Cohen suggested general guidelines 
for levels of effect size. These are .2 for small effect, .5 for medium effect, and .8 for large 
effect. 



A broader debate on the use of statistical significance testing emerged from Cohen's 
power analysis books and other works. Kaufinan (1998) indicates that the "controversy about 
the use or misuse of statistical significance testing has been evident in the literature for the past 
10 years and has become the major methodological issue of our generation" (p. 1). The debate 
has ranged from those who recommend the elimination of statistical significance testing (e.g.. 
Carver, 1978, 1993; Nix & Barnette, 1998) to those who staxmchly support it (e.g., Frick, 1996; 
Levin, 1993, 1998; McLean & Ernest, 1998). However, even those who defend statistical 
significance testing indicate that significant results should be accompanied by a measure of 
practical significance. The leading method of reporting practical significance is through the 
provision of an effect size estimate (Kirk, 1996; McLean & Ernest, 1998; Robinson & Levin, 
1997; Thompson, 1993). Unfortunately, the criteria forjudging the practical significance of 
results based on effect size has defaulted to the use of Cohen's (1988) guidelines that even Cohen 
warned us about (1977, 1988, 1990). 




3 



Protected Effect Size 



3 



Methodology 

Monte Carlo methods were used to generate the data for this research using random 
normal deviates as the basis for sample means to be compared using one-way ANOVA. 
Standardized effect sizes were generated for 5,000 replications within each combination of 
number of groups from 2 to 10 and sample sizes from 5 to 100 in steps of 5, resulting in 900,000 
total replications. The standardized effect size was computed as the range of means divided by 
the root mean square error. In addition, the probability of the observed F statistic was evaluated 
using alpha values of .25, .10, .05, .01, and .001 so each observed effect size could be identified 
as being statistically significant in addition to achieving the general effect size criteria of .2 for a 
small effect size, .5 for a medium effect size, and .8 for a large effect size. Data were generated 
using a program written in double-precision Quick-BASIC. Analysis of the raw data was 
conducted using several routines of SAS®. 



Results 

General Properties of the Distribution of Standardized Effect Sizes 

Table 1 presents the descriptive statistics for the distribution of standardized effect sizes 
for the total set of replications across the range of sample sizes and number of samples. Overall, 
the mean standardized effect size was 0.40639 (SD= 0.29182) with a range of 0 to 5.05513. The 
median standardized effect size was 0.34606. The distribution, as displayed in Figure 1, is 
clearly positively skewed (2.19932) and leptokurtic (7.76283). 

As indicated in the total row of Table 2, more than 80% of the standardized effect sizes 
equaled or exceeded .2, the criterion for a small effect; 24.6% equaled or exceeded .5, the 
criterion for a medium effect size; 8.3% equaled or exceeded .8, the criterion for a large effect 
size; and 4.6% equaled or exceeded an effect size of 1.0. Clearly, it is not at all imusual to get a 
standardized effect size meeting the “small” effect size criterion, about one out of four meeting 
the “medium” effect size criterion, and almost one out of ten meeting the “large” effect size 
criterion. 

Table 2 presents the proportion of those standardized effect sizes achieving selected 
criterion levels by number of samples. It is clear that as the number of samples increases, the 
proportion of standardized effect sizes achieving these cutoffs increases. Table 3 presents 
similar results but for differences in sample size. As sample size increases, the proportion of 
standardized effect size achieving the cutoffs decreases. Based on these trends, it could be 
predicted that the largest standardized effect sizes will be found in the condition of larger 
numbers of smaller sample sizes and the lowest standardized effect sizes will be found in the 
condition of a smaller number of larger sample sizes. 

This prediction is verified in Table 4. The highest standardized effect sizes are found in 
the number of groups equals ten and sample size is 5 where the mean standardized effect size 
was 1.398 with a range of .427 to 3.184. The lowest standard effect sizes were found in the two- 
sample (Ltest) with large sample sizes where the mean standardized effect size was 0.1 12 with a 
range of 0 to 0.500. It is interesting to note that in the two-sample situation, with low sample 
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size (10 or less) it is not at all unusual to get a standardized effect size well above the small 
criterion point. 

If nothing else is clear at this point, it should be obvious that the current arbitrary and 
absolute criteria proposed by Cohen are far from being sensitive to the variation in standardized 
effect sizes as functions of the number and size of samples. Barnette and McLean (1999) 
examined this issue and provided preliminary equations which could be used to predict the mean 
standardized effect size that would occur by chance as functions of number of samples and 
sample size. They concluded that comparisons of observed effect sizes could be made using 
empirically determined standards based on characteristics such as number of samples and sample 
sizes rather than using fixed, arbitrary standards. 

Use of the Significance Test as Protection 

The primary question for this research is: Does the use of a significance test provide a 
way of reducing determination of spuriously high SES values as being meaningful? This issue 
will be addressed in two ways. First, an examination of proportions of effect sizes achieving or 
not achieving the three commonly used effect size criteria and being associated or not with 
statistical significance for the total of number and sample size configuration will be presented. 
Second, a more extensive analysis including number of sample and sample size combinations 
will be presented in the situation where alpha is set at .05 and the medium effect size criterion is 
used. 



Table 5 presents the proportions of standardized effect sizes which do or do not achieve 
statistical significance at alpha levels of .5, .25, .10, .05, .01, and .001 and do or do not 
achieve/exceed the criteria for small, medium, and large effect sizes. In addition, results in Table 
5 provide an indication of the reduction in effect sizes that could be considered to be meaningful 
when a significance test is appUed as an additional criterion for judging effect size. In the 
following discussion, the term “protected effect size” will represent standardized effect sizes that 
meet the dual criteria of achieving the effect size criterion and statistical significance. 

In the situation where the small effect size criterion is used, the proportion of 
standardized effect sizes achieving or exceeding .2 was .8037. Even the use of a significance test 
with alpha of .5 results in the proportion of 40.8% fewer than what might be determined as 
meaningful when no significance test is applied. The protected effect size proportion is .4758. 
When a significance test using an alpha of .25 is applied, there is a lower proportion by 69.1% 
and the protected effect size proportion is .2483. In these two cases, there were standardized 
effect sizes that were significant but did not achieve the small effect size criterion. However, this 
was a small proportion (.0235 when alpha was .5 and .0021 when alpha was .25). In every case, 
for the lower alpha levels, there were no incidences of standardized effect sizes that were 
statistically significant, but did not achieve the small effect size criterion. For the alpha levels 
less than .25, the protected effect size proportion was equal to alpha. 

When alpha was set at .10, application of this as a criterion reduced the meaningful effect 
sizes by 87.6% with a protected effect size proportion of .1000. When alpha was set at .05, 

.1962 of the effect sizes were not statistically significant and did not achieve criterion level. 
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.7536 of the effect sizes were not significant but achieved the small effect size criterion, and all 
of those significant at .05 also achieved the criterion level with a proportion of .0501. These 
proportions are displayed in Figure 2. In reviewing Figure 2, it is important to remember that the 
standardized effect size is positively skewed and the two-dimensional view does not reflect that 
fact. In the .05 situation there was a reduction of 93.8% when the significance test was applied 
in addition to the small effect size criterion. When alpha was set at .01, there was a reduction of 
98.8%; and when alpha was set at .001, there was a reduction of 99.9%. Thus, when looking for 
small effect sizes it is very easy to find them by chance in the ANOVA situation, but the use of 
at leEist a significance test with alpha set at . 1 or lower removes about 88% or more of these 
effect sizes as potentially meaningful differences using a dual criteria decision model. 

In the situation where the medium effect size criterion is used, the proportion of 
standardized effect sizes achieving or exceeding .5 was .2457. Even the use of a significance test 
with alpha of .5 results in the proportion of 24.3% fewer than what might be determined as 
meaningful when no significance test is applied. The protected effect size proportion is .1861. 
When using a significance test with an alpha of .25, there is a lower proportion by 50.9% and the 
protected effect size proportion is .1206. When alpha was set at .10, application of this as a 
criterion reduced the meaningful effect sizes by 75%. 

When alpha was set at .05, .7395 of the effect sizes were not statistically significant and 
did not achieve criterion level, .2104 of the effect sizes were not significant but achieved the 
medium effect size criterion, .0148 were significant but did not achieve the medium effect size 
criterion, and .0353 were protected effect sizes. These proportions are displayed in Figure 3. In 
reviewing Figure 3, it is important to remember that the standardized effect size is positively 
skewed and the two-dimensional view does not reflect that fact. In the .05 situation there was a 
reduction of 85.6% when the significance test was applied in addition to the small effect size 
criterion. 

When alpha weis set at .01, there was a reduction of 96.4% with a protected effect size 
proportion of .0088; and when alpha was set at .001, there was a reduction of 99.6% with a 
protected effect size proportion of .0010, the value of alpha. Thus, in the alpha of .001 situation, 
the proportion of effect sizes that were significant at .001 and did not also achieve the medium 
effect size criterion weis zero. When looking for medimn effect sizes, it is relatively easy to find 
them by chance (about .25) in the ANOVA situation, but the use of a significance test with alpha 
set at .1 or lower removes 75% or more of these effect sizes as potentially meaningful 
differences using a dual criteria decision model. 

In the situation where the large effect size criterion is used, the proportion of standardized 
effect sizes achieving or exceeding .8 was .0834. Even the use of a significance test with alpha 
of .5 results in the proportion of 20.4% fewer than what might be determined as meaningful 
when no significance test is applied. The protected effect size proportion is .0664. When using a 
significance test with an alpha of .25, there is a lower proportion by 46.8% and the protected 
effect size proportion is .0444. When alpha was set at .10, application of this as a criterion 
reduced the meaningful effect sizes by 72.1%. 
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When alpha was set at .05, .8800 of the effect sizes were not statistically significant and 
did not achieve criterion level, .0699 of the effect sizes were not significant but achieved the 
large effect size criterion, .0366 were significant but did not achieve the large effect size 
criterion, and .0135 were protected effect sizes. These proportions are displayed in Figure 4. In 
reviewing Figure 4, it is important to remember that the standardized effect size is positively 
skewed and the two-dimensional view does not reflect that feet. In the .05 situation there was a 
reduction of 83.8% when the significance test was applied in addition to the small effect size 
criterion. 

When alpha was set at .01, there was a reduction of 95.7% with a protected effect size 
proportion of .0036; and when alpha was set at .001, there was a reduction of 99.4% with a 
protected effect size proportion of .0005. Thus, when looking for large effect sizes, about 9% 
may be found by chance in the ANOVA situation, but the use of at least a significance test with 
alpha set at .1 or lower removes 72% or more of these effect sizes as potentially meaningful 
differences using a dual criteria decision model. 

Based on this analysis, it is clear that the use of the dual criteria model may rule out many 
standardized effect sizes as being meaningful when they could easily be attributed to chance. 
Even when selecting a very liberal value of alpha such as .25 or greater, there is evidence of a 
reduction of spuriously high standardized effect sizes. However, it is recommended that an alpha 
of .10 or lower be used in conjunction with attainment of an effect size criterion. 

Since it is clear the standardized effect which may be observed by chance is a function of 
number of samples and sample size, the final part of this analysis looks at one of the situations, 
the medium standardized effect size with alpha set at .05, relative to the need for using a dual 
criteria decision model. Table 6 presents the proportions not achieving or achieving the medium 
effect size criterion and not significant or significant by number of sanlples and sample size. 
Examination of this table provides evidence that, as could be expected from the earlier results, 
the high number of small samples is the combination most in need of such a dual criteria. The 
small number of small samples situation is also in need of such a dual criteria model, but less so. 
Of all of the combinations, those in least need of such an adjustment are those with large sample 
sizes. In these, the number of samples is not as much a fector as when small sample sizes are 
used. Figure 5 presents the distributions of effect sizes for the four smallest-largest numbers of 
samples with the smallest-largest sample sizes. It could be assumed that similar findings would 
result from looking at the other combinations of effect size criteria and alpha levels. 

Conclusion and Implication 

The significance test does provide some protection against judging spuriously high 
standardized effect size values as being meaningful. Applying a statistical significance test does 
substantially reduce the proportion of standardized effect sizes that achieve small, medium, or 
large criteria levels by chance. This is clearly needed in cases where large numbers of small 
samples are used in experimental situations. A level of significance of .1 or lower does provide 
some minimal risk of committing a Type I error while reducing the number of standardized 
effect sizes that would be higher than criterion levels by chance. 
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Perhaps the arguments should not be in favor of using either statistical significance 
testing or effect sizes, but in finding ways, such as this, to combine the philosophies and methods 
of both to make decisions about meaningfiil group differences. It is certainly conceivable that a 
new probability could be determined that would use the combination of effect size and statistical 
significance under given conditions such as the number of samples and sample sizes. It should 
be possible to determine the probability of getting a given effect size and significance level, or a 
protected effect size probability, when the number of samples and sample sizes are known. 
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Table 1. Summary Statistic for Standardized Effect Size for Number of Samples from 2 to 10 and Sample Sizes of 5 
to 100 in Steps of 5 



n 


900,000 


Mean 


0.40639 


Median 


0.34060 


Mode 


0.26625 


Standard Deviation 


0.29182 


Minimum 


0.00000 


Maximum 


5.05513 


Skewness 


2.19932 


Kurtosis 


7.76283 


Standard Error of Mean 


0.00031 
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Standardized Effect Size 

Figure 1. Frequency Polygon of Random Standardized Eff^ Sizes for 2-10 and n= 5-100 
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Table 4. Standardized EflFect Size Statistics by Number of Samples and Sample Size, 5000 Replications 
per Cell 



n 


Statistic 


k=2 


k=3 


k=4 


pr 

II 


k=6 


00 


k= 10 


Total* 




M 


0.5596 


0.8171 


0.9650 


1.0814 


1.1685 


1.2949 


1.3984 


1.0987 




SD 


0.4662 


0.4699 


0.4593 


0.4475 


0.4298 


0.4111 


0.3924 


0.5068 


5 


Min. 


0.0001 


0.0166 


0.0488 


0.1526 


0.0976 


0.2329 


0.4274 


0.0001 




Max. 


3.5991 


5.0551 


3.5266 


4.0714 


3.3395 


3.1073 


3.1844 


5.0551 




M 


0.3752 


0.5475 


0.6614 


0.7445 


0.8106 


0.9041 


0.9818 


0.7601 


10 


SD 


0.2917 


0.2979 


0.2996 


0.2901 


0.2882 


0.2766 


0.2641 


0.3416 


Min. 


0.0001 


0.0073 


0.0453 


0.0772 


0.1303 


0.1619 


0.2240 


0.0001 




Max. 


1.9324 


2.0711 


2.1525 


2.2750 


2.1774 


2.3311 


2.1538 


2.3311 




M 


0.2997 


0.4454 


0.5485 


0.6070 


0.6592 


0.7372 


0.7975 


0.6193 


15 


SD 


0.2310 


0.2399 


0.2396 


0.2337 


0.2251 


0.2172 


0.2146 


0.2752 


Min. 


0.0001 


0.0070 


0.0320 


0.0303 


0.0822 


0.1180 


0.2088 


0.0001 




Max. 


1.6814 


1.6528 


1.6990 


1.8542 


1.5966 


1.8071 


1.7669 


1.8542 




M 


0.2635 


0.3868 


0.4698 


0.5220 


0.5685 


0.6382 


0.6932 


0.5346 


20 


SD 


0.2026 


0.2060 


0.2034 


0.2000 


0.1951 


0.1882 


0.1844 


0.2358 


Min. 


0.0002 


0.0034 


0.0249 


0.0452 


0.0623 


0.1257 


0.1828 


0.0002 




Max. 


1.1976 


1.3665 


1.4187 


1.5536 


1.4428 


1.5091 


1.5889 


1.5889 




M 


0.2303 


0.3365 


0.4141 


0.4674 


0.5080 


0.5738 


0.6137 


0.4764 


25 


SD 


0.1777 


0.1804 


0.1788 


0.1761 


0.1728 


0.1636 


0.1629 


0.2110 


Min. 


0.0000 


0.0045 


0.0337 


0.0669 


0.0877 


0.1561 


0.1541 


0.0000 




Max. 


1.1255 


1.2940 


1.2445 


1.3300 


1.3269 


1.3141 


1.3641 


1.3641 




M 


0.2132 


0.3112 


0.3834 


0.4281 


0.4667 


0.5202 


0.5644 


0.4358 


30 


SD 


0.1636 


0.1699 


0.1666 


0.1639 


0.1589 


0.1521 


0.1486 


0.1926 


Min. 


0.0000 


0.0021 


0.0233 


0.0487 


0.0793 


0.1192 


0.1336 


0.0000 




Max. 


1.1480 


1.2115 


1.1089 


1.2140 


1.1627 


1.2639 


1.2535 


1.2639 




M 


0.1802 


0.2666 


0.3281 


0.3696 


0.4026 


0.4483 


0.4877 


0.3755 


40 


SD 


0.1371 


0.1416 


0.1438 


0.1397 


0.1350 


0.1298 


0.1279 


0.1657 


Min. 


0.0000 


0.0029 


0.0213 


0.0377 


0.0259 


0.1163 


0.1110 


0.0000 




Max. 


0.9093 


1.0157 


0.9659 


0.9683 


0.9746 


1.0279 


1.0421 


1.0421 




M 


0.1446 


0.2203 


0.2682 


0.3017 


0.3274 


0.3684 


0.3943 


0.3066 


60 


SD 


0.1099 


0.1171 


0.1167 


0.1137 


0.1137 


0.1070 


0.1023 


0.1355 


Min. 


0.0000 


0.0072 


0.0068 


0.0280 


0.0284 


0.1060 


0.1105 


0.0000 




Max. 


0.7297 


0.7194 


0.9844 


0.8214 


0.8141 


0.8684 


0.8466 


0.9844 




M 


0.1257 


0.1901 


0.2316 


0.2608 


0.2848 


0.3181 


0.3455 


0.2658 


80 


SD 


0.0963 


0.0996 


0.1014 


0.0970 


0.0961 


0.0917 


0.0906 


0.1173 


Min. 


0.0000 


0.0010 


0.0085 


0.0234 


0.0347 


0.0772 


0.1030 


0.0000 




Max. 


0.6620 


0.6924 


0.7422 


0.7442 


0.6880 


0.7648 


0.7228 


0.7648 




M 


0.1118 


0.1685 


0.2071 


0.2328 


0.2527 


0.2840 


0.3090 


0.2371 


100 


SD 


0.0841 


0.0881 


0.0896 


0.0869 


0.0848 


0.0823 


0.0796 


0.1044 


Min. 


0.0000 


0.0025 


0.0104 


0.0239 


0.0352 


0.0720 


0.1032 


0.0000 




Max. 


0.5002 


0.6262 


0.6363 


0.6667 


0.7125 


0.6244 


0.6319 


0.7125 




M 


0.1972 


0.2924 


0.3552 


0.3987 


0.4338 


0.4852 


0.5253 


0.4064 


Total* 


SD 


0.2068 


0.2381 


0.2563 


0.2691 


0.2789 


0.2943 


0.3082 


0.2918 


Min. 


0.0000 


0.0006 


0.0044 


0.0141 


0.0161 


0.0322 


0.0779 


0.0000 




Max. 


3.5991 


5.0551 


3.5266 


4.0714 


3.3395 


3.1073 


3.1844 


5.0551 



* Totals are based on K of 2 through 10 and n of 5 throu^ 100 in steps of 5. 
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Table 5. Proportions Not Adiieving or Achieving Statistical Significance by Not Achieving or Achieving 
EflFect Size Crit^ion 



Alpha Sign. EflFect Size Crit^ion 


EflFect Size 


Small (.20) 
p(ach.)= .8038 


Medium (.50) 
p(ach.)= .2457 


Large (.80) 
p(ach.)= .0834 


Not Achieved 

p>.50 

Achieved 

50 Not Achieved 

p < .50 

Achieved 


.1727 

.3279 

.0235 

.4758 


.4410 

.0596 

.3133 

.1861 


.4837 

.0170 

.4329 

.0664 


Percent reduction using sign, test 
as additional criterion 


40.8 


24.3 


20.4 


Not Achieved 

p>.25 

Achieved 

9 c Not Achieved 

p < .25 

Achieved 


.1941 

.5555 

.0021 

.2483 


.6246 

.1251 

.1297 

.1206 


.7106 

.0390 

.2060 

.0444 


Percent reduction using sign, test 
as additional criterion 


69.1 


50.9 


46.8 


Not Achieved 

p >.10 

Achieved 

10 Not Achieved 

p <.10 

Achieved 


.1962 

.7037 

.0000 

.1000 


.7157 

.1842 

.0386 

.0614 


.8398 

.0601 

.0768 

.0233 


Percent reduction using sign, test 
as additional criterion 


87.6 


75.0 


72.1 


Not Achieved 

p>.05 

Achieved 

nr Not Achieved 

A U- ^ 

Achieved 


.1962 

.7536 

.0000 

.0501 


.7395 

.2104 

.0148 

.0353 


.8800 

.0699 

.0366 

.0135 


Percent reduction using sign, test 
as additional criterion 


93.8 


85.6 


83.8 


Not Achieved 

p >.01 

Achieved 

Ai Not Achieved 

p <.01 

Achieved 


.1962 

.7937 

.0000 

.0101 


.7530 

.2369 

.0013 

.0088 


.9101 

.0798 

.0065 

.0036 


Percent reduction using sign, test 
as additional crit^ion 


98.8 


96.4 


95.7 


Not Achieved 

P^.OOl 

Achieved 

AAi Not Achieved 

p <.001 

Achieved 


.1962 

.8027 

.0000 

.0010 


.7543 

.2447 

.0000 

.0010 


.9161 

.0829 

.0005 

.0005 


Percent reduction using sign, test 
as additional crita*ion 


99.9 


99.6 


99.4 




20 



Protected Effect Size 



17 



Table 6. Proportion of Mean Standardized Effect Sizes Not Achieving or Achieving or Exceeding 
‘‘Medium Effect Size Criterion” and Not Significant or Significant at Alpha by Number of Samples and 
Sample Size, 5000 Replications per Cell 



n 


Medium Effect Size (.50) 


Sign. 


k=2 


k=3 


k=4 


k=5 


k= 6 


k=8 


k= 10 


Total* 


5 


Not Achieved 


p ^ .05 
p < .05 


.5454 

.0000 


.2718 

.0000 


.1418 

.0000 


.0720 

.0000 


.0316 

.0000 


.0092 

.0000 


.0014 

.0000 


.1214 

.0000 


Achieved or Exceeded 


p2:.05 

p<.05 


.4038 

.0508 


.6798 

.0484 


.8082 

.0500 


.8786 

.0494 


.9220 

.0464 


.9434 

.0474 


.9522 

.0464 


.8298 

.0489 


10 


Not Achieved 


p ^ .05 
p < .05 


.7128 

.0000 


.4942 

.0000 


.3288 

.0000 


.2070 

.0000 


.1320 

.0000 


.0504 

.0000 


.0172 

.0000 


.2277 

.0000 


Achieved or Exceeded 


p ^ .05 
p < .05 


.2370 

.0502 


.4562 

.0496 


.6188 

.0524 


.7450 

.0480 


.8166 

.0514 


.9022 

.0474 


.9376 

.0452 


.7228 

.0495 


15 


Not Achieved 


p ^ .05 
p < .05 


.8166 

.0000 


.6380 

.0000 


.4602 

.0000 


.3506 

.0000 


.2598 

.0000 


.1356 

.0000 


.0700 

.0000 


.3345 

.0000 


Achieved or Exceeded 


p ^ .05 
p < .05 


.1360 

.0474 


.3118 

.0502 


.4840 

.0558 


.6010 

.0484 


.6932 

.0470 


.8192 

.0452 


.8786 

.0514 


.6154 

.0502 


20 


Not Achieved 


p ^ .05 
p < .05 


.8690 

.0000 


.7322 

.0000 


.5944 

.0000 


.4910 

.0000 


.3892 

.0000 


.2376 

.0000 


.1480 

.0000 


.4398 

.0000 


Achieved or Exceeded 


p ^ .05 
p < .05 


.0724 

.0586 


.2198 

.0480 


.3568 

.0488 


.4596 

.0494 


.5618 

.0490 


.7158 

.0466 


.7980 

.0540 


.5101 

.0500 


25 


Not Achieved 


p ^ .05 
p < .05 


.9170 

.0000 


.8142 

.0000 


.7006 

.0000 


.6018 

.0000 


.5094 

.0000 


.3490 

.0000 


.2540 

.0000 


.5410 

.0000 


Achieved or Exceeded 


p ^ .05 
p < .05 


.0332 

.0498 


.1414 

.0444 


.2554 

.0440 


.3458 

.0524 


.4420 

.0486 


.6028 

.0482 


.6974 

.0486 


.4107 

.0483 


30 


Not Achieved 


p ^ .05 
p < .05 


.9390 

.0000 


.8602 

.0000 


.7630 

.0000 


.6968 

.0000 


.6076 

.0000 


.4698 

.0000 


.3592 

.0000 


.6288 

.0000 


Achieved or Exceeded 


p ^ .05 
p <.05 


.0090 

.0514 


.0822 

.0578 


.1820 

.0550 


.2498 

.0534 


.3432 

.0492 


.4844 

.0458 


.5926 

.0482 


.3199 

.0513 


40 


Not Achieved 


p ^ .05 
p < .05 


.9502 

.0210 


.9328 

.0008 


.8770 

.0008 


.8300 

.0000 


.7730 

.0000 


.6722 

.0000 


.5632 

.0000 


.7715 

.0025 


Achieved or Exceeded 


p ^ .05 
p < .05 


.0000 

.0288 


.0192 

.0472 


.0686 

.0536 


1218 

.0482 


.1748 

.0522 


.2808 

.0470 


.3848 

.0520 


.1782 

.0478 


60 


Not Achieved 


p > .05 
p < .05 


.9514 

.0418 


.9472 

.0334 


.9470 

.0174 


.9390 

.0104 


.9144 

.0064 


.8772 

.0060 


.8474 

.0032 


.9092 

.0140 


Achieved or Exceeded 


p ^ .05 
p < .05 


.0000 

.0068 


.0000 

.0194 


.0004 

.0352 


.0132 

.0374 


.0300 

.0492 


.0722 

.0446 


.1072 

.0422 


.0401 

.0366 


80 


Not Achieved 


p ^ .05 
p < .05 


.9474 

.0508 


.9538 

.0408 


.9418 

.0488 


.9540 

.0342 


.9466 

.0284 


.9436 

.0232 


.9276 

.0154 


.9446 

.0315 


Achieved or Exceeded 


p ^ .05 
p < .05 


.0000 

.0018 


.0000 

.0054 


.0000 

.0094 


.0000 

.0118 


.0004 

.2460 


.0052 

.0280 


.0222 

.0348 


.0048 

.0191 


100 


Not Achieved 


p ^ .05 
p < .05 


.9530 

.0468 


.9532 

.0462 


.9476 

.0490 


.9514 

.0454 


.9508 

.0442 


.9452 

.0458 


.9412 

.0432 


.9492 

.0444 


Achieved or Exceeded 


p ^ .05 
p < .05 


.0000 

.0002 


.0000 

.0006 


.0000 

.0034 


.0000 

.0032 


.0000 

.0005 


.0000 

.0090 


.0014 

.0142 


.0002 

.0062 


T 

0 

t 

a 

1* 


Not Achieved 


p > .05 
p < .05 


.9054 

.0276 


.8527 

.0209 


.8010 

.0180 


.7631 

.0144 


.7241 

.0132 


.6642 

.0106 


.6129 

.0082 


.7395 

.0148 


Achieved or Exceeded 


p > .05 
p < .05 


.0446 

.0225 


.0980 

.0284 


.1472 

.0338 


.1875 

.0349 


.2256 

.0372 


.2863 

.0389 


.3372 

.0418 


.2104 

.0353 



* Totals are based on K of 2 through 10 and n of 5 through 100 in steps of 5. 
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